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Abstract. We derive a class of inertial dynamics for games and constrained op- 
timization problems over simplices by building on the well-known "heavy ball 
with friction" method. In the single-agent case, the dynamics are generated by 
endowing the problem's configuration space with a Hessian-Riemannian struc- 
ture and then deriving the equations of motion for a particle moving under the 
influence of the problem's objective (viewed as a potential field); for normal form 
games, the procedure is similar, but players are instead driven by the unilateral 
gradient of their payoff functions. By specifying an explicit Nash-Kuiper embed- 
ding of the simplex, we show that global solutions exist if and only if the interior 
of the simplex is mapped isometrically to a closed hypersurface of some ambi- 
ent Euclidean space, and we characterize those Hessian-Riemannian structures 
which have this property. For such structures, low-energy solutions are attracted 
to isolated minimizers of the potential, showing in particular that pure Nash 
equilibria of potential games are attracting; more generally, when the game is not 
a potential one, we establish an inertial variant of the folk theorem of evolution- 
ary game theory, showing that strict Nash equilibria attract all nearby strategy 
profiles. 



1. Introduction 

The vast majority of problems in mechanical control and classical mechanics 
are governed by the "heavy ball with friction" incarnation of Newton's second 
law: 

x = - grad V - rjx, (HBF) 
where the smooth function V: MP — > 1R represents the system's potential and 
t] > is a friction coefficient which dampens the system and controls the rate 
of energy dissipation. The physical principle of potential energy minimization 
then states that the stable - and, if t] > 0, also attracting - states of (HBF) are the 
local minimizers of V, i.e. the states where the system cannot dissipate any more 
potential energy as heat. 

Even though this principle is not a precise mathematical statement (and does 
not always hold either), treating (HBF) as a second order gradient-like system has 
given rise to an extensive literature in optimization - see e.g. [2, 3, 5, 7, 13, 25] 
and references therein. To wit, Attouch, Goudou & Redont showed in [7] that the 
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inertial system (HBF) can be used for the global exploration of the local minima 
of V: by giving the system enough energy one can escape the attraction of a local 
minimum and converge instead to the global minimum of V. What's more, if 
V is convex and bounded from below, Alvarez showed in [2] that the principle 
of energy minimization holds globally, i.e. the solutions of (HBF) converge to a 
minimizer of V from all initial conditions. 

In this paper, our primary motivation is to extend these second order, inertial 
methods to mixed strategy learning in games, viewed here as concurrent, multi- 
agent, multi-objective optimization problems over products of simplices. The first 
difficulty in this endeavor is that (HBF) cannot be immediately constrained to a 
compact polytope X of R", even if the driving force F = — grad V vanishes on the 
boundary bd(X) of X: if a solution trajectory of (HBF) starts with a sufficiently 
high velocity, then it will escape X in finite time. In view of this, Flam and Morgan 
studied in [12] a projected variant of (HBF) where trajectories were constrained to 
X by projecting to zero any inadmissible velocity components at the boundary of 
X. However, as a result of this projection, the dynamics of [12] end up not being 
Lipschitz, so even existence and uniqueness results are hard to obtain. 

To avoid well-posedness problems of this sort, we construct an interior-point 
learning method by endowing the problem's state space X with a Riemannian 
metric g which blows up at the boundary of X, and then deriving the associated 
covariant version of (HBF): the metric's blow-up will slow down trajectories near 
the boundary of X, so the resulting system is expected to be well-posed. In first 
order, this idea was exploited by Alvarez, Brahic & Bolte [4] who studied the 
gradient flow x = — grad V for a general class of Hessian-Riemannian metrics 
obtained by taking the Hessian g = Hess (ft) of a Legendre-type function h on X, 
i.e. a strictly convex function whose derivative blows up at the boundary of X: 
with grad V vanishing on bd(X), the corresponding gradient flow stays in X for 
all time and converges to a minimizer of V when V is convex [4]. 

To apply this Hessian-Riemannian approach to a second order optimization 
setting, the acceleration x in (HBF) must also be replaced by its covariant ana- 
logue, viz. the covariant derivative D 2 x/Dt 2 = V^x of a curve's velocity along 
itself. To that end, we calculate the Levi-Civita connection of the Hessian-Rie- 
mannian structure on the simplex and derive an explicit expression for the co- 
variant "heavy ball with friction" dynamics 

^J = -grad g y- f7 x, (HBF*) 

i.e. the equations of motion for a particle moving under the influence of the 
system's objective function (viewed as a potential field). These dynamics are then 
carried over to normal form games by replacing the gradient of the objective 
with the unilateral gradient of each player's individual payoff function, i.e. the 
gradient-like vector obtained by differentiating along the strategies of the focal 
player and keeping the strategies of his opponents fixed. 

In the special case of the Shahshahani metric (corresponding to the Hessian of 
the Gibbs entropy) [28], this approach yields an inertial variant of the well-known 
replicator dynamics of evolutionary game theory [29]. 1 The resulting dynamics 



Similarly, in first order, the Hessian-Riemannian gradient flow corresponding to the Gibbs en- 
tropy function is just the ordinary replicator equation [4]. 
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thus beg to be compared with the second order replicator equation that was 
recently derived in [17] as the offshoot of a higher order exponential learning 
scheme: therein, it was shown that even weakly dominated strategies become 
extinct and that strict Nash equilibria attract all nearby strategy profiles. On the 
other hand (and somewhat surprisingly at that), the inertial replicator dynamics 
exhibit no such properties: despite all intents to the contrary, their solution orbits 
collide with the boundary of X in finite time. 

This phenomenon is due to the fact that the blow-up of the Shahshahani metric 
is not sufficient to contain trajectories within X. Indeed, as was shown by Akin 
[1], the n-dimensional simplex endowed with the Shahshahani metric is isomet- 
ric to an (open) orthant of the n-dimensional sphere. This escape can thus be 
explained by noting that a particle which is constrained to move on the sphere 
under the influence of a finite force will escape any orthant of the sphere in finite 
time. Extending this reasoning, we derive an explicit Nash-Kuiper embedding of 
the simplex which allows us to characterize those Hessian-Riemannian structures 
for which (HBF) is defined for all time: if the interior of the game's strategy space 
X can be mapped isometrically to a closed hypersurface of some ambient Eu- 
clidean space, then the trajectories of the covariant version of (HBF) will remain 
in X for all time. 

When this is the case (the metric generated by the Hessian of the log-barrier 
function on the simplex being a prime example), we show that isolated mini- 
mizers of the potential attract all nearby solutions which start with low enough 
energy. On the other hand, as far as games are concerned, we establish in Section 
5 the following inertial variant of the folk theorem of evolutionary game theory 
[15, 27] and we show that: a) Nash equilibria are stationary; b) if an interior orbit 
converges, its limit is a restricted equilibrium; c) if every neighborhood of a point 
admits an interior orbit which remains in the neighborhood in question for all 
time, this point is a restricted equilibrium; and, finally, d) strict equilibria attract 
all nearby strategy profiles. 

Paper outline. In Section 2, we give a brief account of the geometric machinery 
that we will need along with some elementary definitions from game theory. Our 
analysis proper begins in Section 3 where we introduce a class of (decompos- 
able) Hessian-Riemannian metrics with respect to which we derive the covariant 
analogue of (HBF) for the simplex; we also derive there an isometric embed- 
ding of the n-dimensional simplex in Euclidean space which allows us to view 
the derived dynamics as a classical mechanical system constrained to move on 
a hypersurface of ]R n+1 . Section 4 is then devoted to single-agent, constrained 
optimization, while Section 5 contains our game-theoretic results; finally, in Sec- 
tion 6 we show that the inertial replicator dynamics do not admit global solutions 
(so the analysis of Sections 4 and 5 does not apply), and we characterize those 
Hessian-Riemannian structures that do. 

2. Notation and Preliminaries 

In this section our aim will be to fix our game-theoretic notation and to review 
some basic notions from Riemannian geometry, mostly following the excellent 
reference [18] for the latter. 
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2.1. Notational conventions. Let § = {s a }" =0 be a finite set. The real vector 
space spanned by S will be the set of formal linear combinations of elements of § 
with real coefficients, i.e. the set R s of all maps x : S — >■ R, s 6 S i— > x s G R. The 
canonical basis {e s }seS or this space consists of the indicator functions e s : §->R 
which take the value e s (s) = 1 on s and vanish otherwise, so thanks to the natural 
identification s t-> e s , we will make no distinction between the elements s of § 
and the corresponding basis vectors e s of R s . In fact, we will use a to refer 
interchangeably to either s a or e a , and the set A(§) of probability measures on 
S will be identified with the n-dimensional simplex A(§) = {x G R s : J^ a x a = 
1 and x a > 0}. 

Regarding players and their actions, we will follow the original convention of 
Nash and employ Latin indices (k, t, . . . ) for players, while keeping Greek ones 
(«, j6, . . . ) for their actions (pure strategies); also, unless mentioned otherwise, we 
will use a, jS, . . . , for indices that start at 0, and k, ji, v, . . . (or Latin characters) 
for those which start at 1. Finally, if {&k}k£X is a family of finite sets, we will 
write (a; for the tuple . . . , a, ajt+i/ • • • ) G I~Lt an d we will use the 
shorthand instead of £ ae s.. 

2.2. Definitions from game theory. A finite game = <S(N,A,u) will be a tuple 
consisting of: d) a finite set of players N = { 1, . . . , N}; b) a finite set ^ of actions (or 
pure strategies) for each player A: G N; and c) the players' payoff functions A — >■ 
R, where ^1 = n^-^i: denotes the game's action space, i.e. the set of all action 
profiles («!, . . . , a N ), G A fc . 

If players mix their actions by taking probability distributions Xfc = (Xfca)«e./U G 
A(yijt) over their action sets Ajt, their expected payoffs will be 

"(tO) = E ai '"D aN M t( a i'---' a N)^ui--- ^n,« n , (2.i) 

where x = (x\, . . . , x^) is the players' (mixed) strategy profile and Myt( a i/ • • • / *n) 
denotes the payoff to player k in the (pure) action profile (ot\, G .A; 

moreover, if player A: plays the pure strategy a G A^, we will use the notation 
u ka{ x ) = u /c( a ' x -/c) — Uk(x\, ,x^). In this mixed context, the strategy 

space of player k will be the simplex Xj- = A(Ai c ) while the strategy space of the 
game will be the convex polytope X = Ylk ^k- Together with the players' (ex- 
pected) payoff functions u^: X — >■ R, the tuple (K, X, u) will be called the mixed 
extension of © and it will also be denoted by (relying on context to resolve any 
ambiguities). 

The most prominent solution concept in game theory is the concept of Nash 
equilibrium which characterizes strategy profiles that are resilient against unilat- 
eral deviations. More formally, we will say that q G X is a Nash equilibrium of & 
when 

u ki x k' a -k) ^ u k(q) ror a H x k £ X/ c and for all k G N, (NE) 
and if the above inequality is strict for all G \ {qk}, k G N, then q will 
be called a stn'cf Nash equilibrium - obviously, by multilinearity, only "pure" 
strategy profiles (viz. the vertices of X) can be strict equilibria. On the other hand, 
if (NE) only holds for Xj. in the sub face X' k spanned by the support supp(^) = 
{oc G A]c : qka > 0} of q^, then q will be called a restricted Nash equilibrium of 25, 
reflecting the fact that it is a Nash equilibrium of the restriction &' = <&\ X i = 
0'(N,X',m| x O of <StoX'. 
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An especially relevant class of finite games is obtained when the players' payoff 
functions satisfy the potential property: 

uUx) - u k p(x) = - {V{ot;x_ k ) - V(0;je_it)) (2.2) 

for some (necessarily) multilinear function V : X — > R. When this is the case, the 
game will be called a potential game with potential function V , and as is well known, 
the pure Nash equilibria of © will be the vertices of X that are local minimizers 
of V [23]. 

2.3. Elements of Riemannian geometry. For notation and as a general reference 
on Riemannian geometry, we refer the reader to the masterful account of [18]; in 
what follows, we simply give a brief overview of the notions that we will use for 
clarity and completeness. 

Let U be an open set of R'" and let p £ U. The tangent space to U at p is 
traditionally defined as the space R™ = {p} x R m = {(p,z) : z £ R'"}; equiv- 
alently, a tangent vector X £ T p U may also be seen as a linear operator acting 
on smooth functions / £ C°°(U) by means of directional differentiation, i.e. as a 

map / i — y X(f) = Jj q/(P + ^0- Dropping the basepoint p for notational con- 
venience, the standard basis of TpU will be denoted as {Ej}JL^, and, depending 
on the context, Ej will be viewed interchangeably as a column vector in R"' or as 
an operator sending / i-> E;(J) = djf\p. 

The cotangent space to U at p will be the dual space T*U = (TpU)* of TpU, 
and the action of a covector (or 1-form) a> £ T*X on X £ TpU will be written in 
Dirac's bra-ket notation as o>(X) = (cv\X). Fibering the above constructions over 
U, we then obtain the tangent (resp. cotangent) bundle TU = Up^uTpU (resp. 
T*U = Upeu TpU)> also, the space of vector (resp. covector) fields on U will be 
denoted by T( U) (resp. 7*(U)). 

An inner product on TpU will be a positive-definite pairing on TpU, i.e. a 
bilinear form gp : TpU x TpU — > R such that: 

(1) g P (X, Y) = g P (Y,X) for all X, Y £ T p U. 

(2) g p (X, X) > with equality if and only if X = 0. 

If there is no doubt as to the metric tensor g in question, we will occasionally use 
the more suggestive notation (X,Y)p = gp[X,Y); in the same spirit, when the 
basepoint p £ U may be inferred from the context, we will drop it altogether and 
simply write (X, Y) instead. The norm of X is then defined as 

||X|| 2 =(X,X) (2.3) 

and the orthogonal projection of X £ TpU on Y £ T p U will be 

Extending these constructions to U, a Riemannian metric will be a smooth as- 
signment of an inner product gp to every p £ U. The components of g in the 
frame {Ej} will then be 

g lj (p)=gp(E„E j ), (2.5) 

so a Riemannian metric may be seen as a smooth field of positive-definite matrices 
on U. From a dual point of view, given that the pairing gp: TpU x TpU — > R 
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is positive-definite (and, hence, non-singular), we may assign to each covector 
co 6 T*U a unique vector 6 T p U such that 

(w", Y) = {co\Y) for all Y e T p U. (2.6) 

The resulting tangent-cotangent isomorphism (j : T*!i — > T p (J is known colloqui- 
ally as the "musical isomorphism", and it may be written in components as 

"HELs'W (2-7) 

where gi k denotes the inverse matrix of g, i.e. 

Lisas'" = (2-8) 

Accordingly, we define the gradient of a smooth function / £ C°°(!i) with respect 
to g as 

grad g /=(d/)«, (2.9) 

or, in components: 

(g-d ? A = EL/^- (2.io) 

As usual, when there is no danger of confusion, grad / will be denoted more 
simply by grad/. 

In view of the above, differentiating a function / E C°°(ii) along a vector field 
X 6 7(U) simply amounts to taking the pairing (grad/, X) = (df\X) = X(/). 
On the other hand, to differentiate a vector field along another, we will need the 
notion of a (linear) connection on U, viz. a map 

V: 7(U) x 7(U) -> 7(U) (2.11) 

written (X, Y) h-S> VxY, and such that: 

(1) V flXl+f2 x 2 Y = /iV Xl Y+/ 2 Vx 2 Yforall/ 1 ,/ 2 e C°°(ll). 

(2) V x (flY! + bY 2 ) = aV x Y 1 + bV x Y 2 for all a, b e R. 

(3) V x (/Y) = / • V X Y + V x / • Y for all / e C°°(U) 7 where V x / = X(/) = 
(d/|X>. 

In this way, VxY generalizes the idea of differentiating Y along X and it will be 
called the covariant derivative ofY in the direction ofX. 

In the standard frame {Ej} of TU, the defining properties of V give 

VxY = E^! + El;, fc=1 1?, (2-13) 

where the Christoffel symbols r| G C°°(!J) of V in the frame {Ey} are defined via 
the equation 

v Ei E ; = ELi r ^- (2.14) 
Clearly, V is completely specified by its Christoffel symbols, so there is no canon- 
ical connection on U; however, if U is also endowed with a Riemannian metric 
g, then there exists a unique connection which is symmetric (i.e. r| = T^) and 
compatible with g in the sense that: 

V X (Y,Z) = (VxY,Z) + (Y,V x Z) for all X, Y,Z e T(U). (2.15) 
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This connection is known as the Levi-Civita connection on U, and its Christoffel 
symbols are given in coordinates by 



ij 2 Li=l8 I dx . -r dx . dX( 



(2.16) 



In view of the above, the covariant derivative of V along a curve 7 on U will be 
^ = V* V ee , =1 (V, + 1* Vfy) £ /c (2.17) 

so, specializing to the case where V(t) is simply the velocity v(t) = 7(f) of 7, the 
acceleration of 7 will be defined as = 137 = or ' m components: 

+ (2.18) 

3. Derivation of the dynamics 

Motivated by the optimization properties of gradient-like dynamics that are 
second order in time [2, 3], our goal in this section will be to derive a covariant 
version of (HBF) for the n-dimensional simplex X ee A(n + 1) of Of course, 

if X is endowed with the standard Euclidean metric (inherited from the ambient 
space IR" +1 in which it resides), then there will be no constraint force restricting 
the dynamics to remain in the simplex and solution orbits will escape in finite 
time. As such, we will instead follow the Hessian-Riemannnian approach of [4] 
and try to raise an inherent geometric barrier at the boundary of X by endowing 
the positive orthant IR"^ 1 ee {x 6 K n+1 ,X; > 0} of R n+1 with a Riemannian 
metric which blows up at the boundary hyperplanes x a = 0. 

3.1. Hessian-Riemannnian metrics. In the context of [4], this blow-up property 
is obtained by letting the metric tensor be the Hessian of an entropy-like func- 
tional that becomes infinitely steep near the boundary of R+of. Specifically, let 
6 : [0, +00) — > IR U { +00} be a C°° function satisfying the Legendre-type properties 
[4,26]: 2 

1. 6(x) < 00 for all x > 0. 

2. lim^ + 6' (x) = -00. (L) 

3. 6"(x) >Oand0'"(x) < for all x > 0. 
Moreover, let 

h(x) = JZ=o 9 M 

and define a metric g on IR"^ 1 by taking the Hessian of h in standard coordinates, 
i.e.: 

d 2 h 

= dx~dx~a = eA ?' (3 ' 2) 



Legendre-type functions are usually defined without the regularity requirement 8"' < 0. This 
assumption can be relaxed without significantly affecting our results but we will keep it as is to 
simplify our presentation. 
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where the shorthand a = 0,...,n, stands for 6"(x x ). In other words, the 
Hessian-Riemannnian metric associated to 8 will be the field of positive-definite ma- 
trices 

g(x)=dmg(8"(x ),...,8"(x n )), (3.3) 
With 8 strictly convex (8"(x) > 0), it follows that g is indeed a Riemannian metric 
on R""^ 1 ; accordingly, when g is derived in this way, 8 will be called the kernel of 
g and h will be its associated entropy function. 

Remark 3.1. The Legendre-type function h of (3.1) is closely related to the control 
cost functions used to define quantal responses in the theory of discrete choice 
[21, 30], so there is a deep connection between the geometry of the simplex and 
the theory of quantal /smoothed best responses; for an extended discussion of 
this issue, see [10, 22] and references therein. 

Remark 3.2. More general Hessian-Riemannian structures can be obtained by 
considering Legendre-type functions h : ]R+"+ — >■ K that do not necessarily admit 
a decomposition of the form h{x) = Yll=o®{ x a)- Most of our analysis can be 
extended to this more general setting, but since the calculations involved are 
significantly more tedious, we have opted to present our analysis in this simpler, 
decomposable setting. 3 

Example 1 (The Shahshahani metric). Perhaps the most well-known example of a 
Hessian-Riemannian structure on the simplex corresponds to the kernel 8$ (x) = 
xlogx which gives the Shahshahani metric [1, 4, 28] 

g s {x) =diag(l/x 0/ ...,l/x„), ^R" + +, (3.4) 

or, in coordinates: 

dp = $ccp/xp, (3-5) 

As the Hessian of the (negative of the) Gibbs entropy h(x) = yj a x a logx a , 4 the 
Shahshahani metric is intimately tied to the well-known replicator dynamics of 
evolutionary game theory [4, 14, 15], so a large part of our analysis will be focused 
on g s and its properties - see [14, 15] and [27] for an excellent discussion on the 
topic. 

Example 2 (The log-barrier). Another noteworthy example with important ties 
to proximal methods in optimization (see [4] and references therein) is given by 
the logarithmic barrier kernel &i(x) — — logx which generates the well-known 
Itakura-Saito divergence [4, 8, 11, 20]. This choice of 8 corresponds to the log- 
barrier function h(x) = — log x a whose Hessian generates the metric 

#4! = s *p/x% ( 3 - 6 ) 

or, in matrix form: 

g L (x) =diag(l/xg 1/xl), JCGRIJ+ 1 . (3.7) 

An important qualitative difference between g s and g L is that the kernel of the 
former is finite everywhere, whereas the kernel of the latter blows up at 0; as we 

In particular, the results that do not hold verbatim are those that call for an explicit expression 
on 9 - most notably, Corollary 6.2. The analysis of Section 3.4 will also carry through by invoking the 
Nash-Kuiper embedding theorem - all that is lost is the explicit expression (EC). 
^This is also the reason that we are calling h the entropy function of g. 
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shall see later, this point will have important consequences with regards to the 
existence of global solutions. 



Now, having endowed R" + 1 with a Riemannian metric g, the interior X° of the 
unit n-dimensional simplex A inherits itself a Riemannian structure by restricting 
g to TX°. For calculations (and to make this idea more explicit), it will be conve- 
nient to introduce (global) coordinates on X° via the surjection ttq : R" +1 — » R" 
which forgets the 0-th component of (xo, X\,..., x n ), i.e.: 

7r (x ,xi,. ..,x„) = (xi,...,x n ). (3.8) 

In this way, if we denote the image of X° under tiq by U = {w 6 R" : w ft > 
and Y^i=i w \i < 1}/ the injective immersion to: U — » X° defined as 

i (w lf ...,w n ) = (1 -E^l^/Wi/.-./Wn) (3.9) 

will be an inverse to Tto\x°/ an d hence a smooth embedding. The standard coor- 
dinate frame 1 of TU corresponding to the coordinates (w\, ...,w n ) of R" 
will thus push forward via /g to the frame E« — £o or TX° in the sense that 

4 (/ °' 0) = |;^ 'o,all/ £ C»( E »«) (3..0) 

As such, the induced Riemannian metric g on (J may be expressed in components 
as: 

g ¥V = g (E,„ £ v ) = g (E,, - E Q , E v - E ) = 0^ + O ', (3.11) 
where ^'(x) = 6"(xp) and O ' is to be understood as 6%(w) = 8"(1 - E F 

3.2. Covariant differentiation and Newton's law. With this coordinate represen- 
tation at hand, the Christoffel symbols T^ v of g will be given by the expression 
= YLp§ Kp ^pftv where, following (2.16), the Christoffel symbols of the first kind 
Tp^ are defined as: 



V= W-^L (3.12) 



1 / dgpp dfpv _ dgjw 

2 \ dw v dw } i dw. 

Even though a direct calculation of fp-jiv from (3.11) is possible if one is armed 
with enough patience, it is simpler to note that the push-forward expression (3.10) 

-Or. ~ 

implies that g^ v = £ w where h = ho l : U —> R is the pull-back of h to U via 
^0- By the equality of mixed partials, we thus obtain: 

d 3 h 1 



^ = 2 dw p dw }l dw v = 2 " ^ ) ' (3 ' 13) 

where now 8p }lv = 5pp6p V denotes the triagonal Kronecker symbol (Sp^y = 1 if p = 
]i = v and otherwise) and d'p, f> = 0,1, . . .,n, is shorthand for O'^'(x) = 9"'(xp). 

To continue, we will need to calculate the inverse of the metric tensor guy = 
O'udfiv + 6'o on U; to that end, we have the following recipe: 

Lemma 3.1. Let A^ v = q^S^y + q$ with q$,q\, . . . ,q n > 0. Then, the inverse A^ v of 

Apiy is 

AV v = s j^_Qh_ > (3 . 14) 
q^i q^qv 
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where denotes the harmonic aggregate Q^ 1 = E«=o %. 1 - 
Proof. By a straightforward verification, we have: 

E"=i A w AVp = T n ,(l^v + qo)(Svp/qv ~ Qk/iWp)) 

= J2 V {q^iivdvp/qv + q<d&v P /qv - q^Qh^v/ '(qvqp) - qoQh/{qvq P )) 

= Spp + qoqp 1 - Qirfp 1 - qoQhqp 1 L v <?v 1 = hp- D 

As a result, by setting 0^' = (j^p 1 / , the inversion formula (3.14) above 
yields 



S <P>" 
5 an qiiqii' 



(3.15) 



and, in combination with (3.13) and (2.16), we finally obtain: 
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~^ 8}lv ~Wl)' (3 ' 16) 

where we used the fact that Ylp=\ = 1 / — 1 /$o m * ne secon d line. Con- 
sequently, the covariant acceleration (2.18) of a curve w(t) on U will be: 

dV_.. i v A. «V . 

, 10" 0'" 2 10'"©'' , n2 /oi ^ 

^'n^-^^^^w^^) ■ (3 - 17) 

Despite the apparent ugliness of this last expression, things become consider- 
ably simpler if we write (3.17) in the original coordinates (xq, X\, . . . ,x n ) of R+lj. 
- i.e. by considering the curve x(t) = io(w(t)). In that case, with xq = 1 — yj^ w-fi, 
we will have YLu ™}t = — *0> an d after some easy algebra, (3.17) gives the following 
expression for the covariant acceleration: 

D 2 x a .. 1 1 



Df 2 



^(«-e;= o ( w)^)- ^ 



Remark 3.3. The cumbersome calculations leading to (3.18) could have been cir- 
cumvented by working directly on IR^ 1 (where the Christoffel symbols of g are 
trivial to calculate), and then "projecting" the resulting second order expressions 
to X by employing the theory of holonomically constrained mechanical systems 
[6, 19] to the equality constraint J^pXp = 1- However, the calculation of the 
resulting constraint force (the summation term in (3.18)) turns out to require a 
comparable amount of work by itself, and since this approach would necessi- 
tate the introduction of further machinery anyway, we chose to follow the more 
straightforward (albeit perhaps less elegant) approach above. 
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Now, to write down Newton's second law of motion for a frictionless heavy 
ball moving in X, it will again be more convenient to work with the variables 
w and then pull back the results to X° via lq. Accordingly, let F G T(IT) be a 
force field on U written in components as F = Ylu=\ fyEp where E f[ denotes the 
standard frame of U. Then, by virtue of the push-forward relation (3.10), the 
equations of motion in the standard coordinates of X° will take the form: 

D 2 x 1 _jP x if«^0, 

Even though the direction a = is not really being discriminated against in 
the above equation, the resulting expression for Newton's law is not particularly 
handy to work with; on the other hand, things become considerably simpler and 
more intuitive in the conservative case where F = — grad V for some smooth 
function V : U — >■ R corresponding to a potential V = V o ttq : X° — > ]R on the 
simplex. In that case, the expression (3.15) for gl lv readily yields: 



i» L Q n dXv + Q „ gii dxQ ~ e ,, y dXfi L ff, dXfi J • 

(3.20a) 

where in the second equality we used the fact that Yai l/0y = 1/©/' — l/#0' wrtn 
some more algebra, we then get the following expression for the last term of 
(3.19): 

_ f p = f -— -(— -\ f ^L^- 

l—i t \ l l—i an ~\y I p)// all j 2-i nil ^ Y 

■±$w)- < 3 - 2 °"> 

In view of the above, we may now write down an explicit expression for the 
Hessian-Riemannian analogue of (HBF) for the simplex. Indeed, starting with 
the equation of motion 

^ = - gra dV- V x, (HBF*) 

and plugging in the covariant acceleration D 2 x/Dt 2 from (3.18) and the gradient 
of V from (3.20), we readily obtain the component-wise inertial dynamics: 

(ID) 

where we have set u a = — d a V and the term —rjx describes a friction force pro- 
portional to the particle's velocity. 

The dynamics (ID) will constitute our main focus, so a few examples are in 
order: 
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Example 3 (The inertial replicator dynamics). In the case of the Shahshahani 
kernel 6(x) = xlogx, we will have 6"(x) = 1/x and 6"'(x) = -1/x 2 , so (ID) 
yields the inertial replicator dynamics: 

x a — x a — y^, p x p u pj + (%«./ x \ ~ Yi p X fi) ~ nx &- (I-RD) 

It is thus interesting to compare this inertial variant of the replicator dynamics in 
the frictionless case {n = 0) with the second order system 

x a = x a (u a - J2p x P u p) + x <* (*«A« - L^/sA/s) ' (2-RD) 

which appeared in [17] as an offshoot of the second order exponential learning 
scheme: 

J/« = u a 

= exp(y ffi ) (3.21) 
E/jexp(y i3 )' 

Remarkably, despite their very different origins (from geometrical and learning 
considerations respectively), the only difference between these two second order 
extensions of the replicator dynamics is the factor 1/2 in the "constraint force" 
term. Even more surprisingly, and in spite of the innocuous appearance of this 
scaling-like difference, 5 we shall see in the following sections that (I-RD) and 
(2-RD) behave in drastically different ways. 

Example 4. In the case of the log-barrier kernel 6(x) = — logx, we will have 
9"(x) = 1/x 2 and 8"'(x) = — 2/r 3 , thus obtaining the inertial log-barrier dynamics: 

x a — x k (yUu ~ T x ^$upj + x 2 (^x 2 /x^ — r 2 x 2 ^/ x^j — t]x K , (I-LD) 

where r 2 = Y^ = o x ^,- The first order analogue of these dynamics, namely the 
system x K = x 2 (u a — r~ 2 x ^ u p) i has been studied extensively in the context 
of linear programming [8, 11, 20] and convex analysis [4]. On the other hand, 
the game-theoretic properties of the first order log-barrier system have not been 
investigated, so it is not easy to make a comparison with the benchmark case 
of the replicator dynamics. 6 As we shall see, (I-RD) and (I-LD) will turn out 
to behave in qualitatively different ways, so the geometry of the simplex plays 
a crucial role in determining the long-term properties of the resulting inertial 
dynamics. 

3.3. Friction and Dissipation of Energy. By the definition of a conservative force, 
it follows that the total energy 

E{x,v) = hv,v) +V(x) (3.22) 

of the system will remain constant along the solutions of (ID) when u a = — 
for some potential function V £ C°°(X ) and f] = (see e.g. [19] for a proof of 
this fact in a general Riemannian setting). On the other hand, for n > 0, we have 
the following dissipation result: 



One might be tempted to interpret the factor 1 /2 in (I-RD) as a change of time with respect to 
(2-RD), but the presence of x 2 precludes as much. 

^For a discussion on this topic, see the forthcoming paper [22]. 
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Proposition 3.2. Let x{t) be a solution orbit of (ID). Then, the total energy E = 
\(x,x) + V(x) of the system will be non-increasing along x(t); in particular, we will 
have: 

E = —2rjK = —i]\\x\\ 2 , (3.23) 
where K = \ ||x|| 2 is the particle's kinetic energy. 
Proof. By differentiating (3.22), we readily obtain: 

£ = Vi-E = lVx{x,x) + ViV = (VxX,x) + (dV\x) 
D 2 x \ 

j^2> x ) + (grad V,x) = (-grad V - nx,x) + (grad V,x) 

= -n(x,x) = -2t]K, (3.24) 

where we used the metric compatibility (2.15) of V in the first line, and the 
definition of the dynamics (ID) in the second. □ 

This proposition essentially shows that, for rj > 0, the system's total energy E 
is a Lyapunov function for (ID). This is the main difference between (ID) and the 
Hessian-Riemannian gradient flows studied in [4]: in the first order regime, it is 
the potential V that plays the role of a Lyapunov function, and not the system's 
mechanical energy E = K + V. To recover the first order context, it suffices to 
include the mass m of the particle in our analysis: by replacing D 2 x/Dt 2 by 
mD 2 x/Dt 2 in (HBF*) and K by mK in (3.22), the first order dynamics of [4] are 
obtained in the dissipative, "light ball" limit m — » 0. 

3.4. Euclidean coordinates. In our derivation of the dynamics (ID), we endowed 
X with a Riemannian structure and then expressed Newton's second law in the 
corresponding covariant form. The advantage of this approach is that the configu- 
ration space of the dynamics retains its original form; on the downside however, 
the geometry of X° is far from Euclidean, so intuition may be led astray and 
explicit calculations are made harder by the coordinate expressions for g. An 
alternative approach (which turns out to be very convenient from a calculational 
standpoint) is to isometrically embed X° in some ambient Euclidean space and 
then write the ordinary (Euclidean) version of Newton's second law for a particle 
constrained to move on this embedding. 



Formally, recalling that X° inherits its Riemannian structure from (R 



n+l 

++ • 



the above boils down to finding a Euclidean embedding <$>: — > K m , i.e. an 
injective immersion of X° such that 

g(E a ,E fi ) =S(^,E a ,^,E fi ) for all a, (3 = 0,1,..., n, (3.25) 

where 5 denotes the standard Euclidean metric of IR'" and ^>*E K is the push- 
forward of E K under <I>, viz. 

®* E «=L^ E P> (3-26) 

with {Ep} denoting the standard basis of the target space 1R" ! . 

Remark 3.4. In what follows, vectors taken in a Euclidean space will be distin- 
guished by an overline (such as E K above); also, we will use "■" to denote the 
ordinary Euclidean inner product and | ■ | for the Euclidean norm. 
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That such an embedding exists is a consequence of the celebrated Nash-Ku- 
iper embedding theorem [16, 24]; however, the Nash-Kuiper theorem does not 
provide an explicit construction method in the general case. Fortunately, in our 
Hessian-Riemannian framework, the situation is considerably simpler: indeed, 
let <p: (0, +oo ) — » IR be smooth and increasing, and consider the embedding 
O: -> with (x 0r ...,x n ) ^ {<p(x ) r ... r <p(x„)). Then, if we denote 

the standard coordinates of the target space by ■ ■ -/£n) an d its corre- 

sponding basis by {£«}^ =0 as above, we will have 0*£ ff = EjS w^Ep = £*• 
Therefore, for O to be an isometry, (3.25) shows that it suffices to have 

gap = 6 ((p'{x a )E a ,(p'{xp)Ep) = 5 K p<p'{x K )<p'{xp), (3.27) 
or, more concisely: 



cp'( x ) = yf6»(x). (3.28) 

Motivated by the above, we will say that the variables £ = <I>(x) with 

S K = <p(x K ) (a = 0,1,..., tt) (EC) 

are Euclidean coordinates for (R"^ 1 ,^) if (p: (0, +oo) — > R satisfies (3.28) for all 
x > 0. Correspondingly, with £ a = (p'(x a )x a = y/6^x a and £ a = 6%'/ {2yJWl)x^ + 
\[&lx.cLr some algebra yields the following expression for the inertial dynamics 
(ID) in Euclidean coordinates: 

& = -L o« - E/3 «*) + * -L E/3 «w/w) z e - (e-id) 



It is then interesting to note the following with regards to (E-ID): 

Remark 3.5. In the case of a conservative force with potential V, we will have 
— — dV-^ — —u a / so the first term of (E-ID) may simply be interpreted as 



3x« dgu 

(minus) the Euclidean gradient of V w.r.t. £ projected on the system's configuration 
manifold: 

S = 0(X°) = {^e]R"+ 1 :^^ 1 (^) = l}. (3.29) 

Indeed, the (Euclidean) gradient N s = grad (E^ -1 ^)) = Lp(d'P~ 1/2 Ep of 
the configuration constraint E^ <? _1 (£^) = 1 wm be normal to S by definition, so 
the projection of gradV = — Ep^) -1 ^ 2 "/?^ on S will be: 

pr s gradV = gradV — pr N gradV 

= -E a (C)- 1/2 ««E«+@;;E^/3/^E a K / )^ 1/2 ^ (3-30) 

which is simply the first term of (E-ID). In other words, the first term of (E-ID) 



represents the tangential force which drives the system along the (projection of the) 
gradient of V on the configuration manifold S. 

Remark 3.6. Orthogonally to the above, the second term of (E-ID) represents the 
centripetal force which constrains the dynamics to remain on the configuration 
manifold S and which is thus normal to S [6, 19]. To see this, note that if £(f) 6 S, 
we will have E^ t P~ 1 {^p) = 1> and hence: 



dt 



£ a <p-\Ui)) = E* = E» t'lyffl = o- (3-3i) 
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Thus, given that the components of the centripetal force are proportional to 
(^a:) -1 " 2 ' it follows that the force itself will be normal to £, and hence to S. In 
classical mechanics, this fundamental property of holonomically constrained sys- 
tems (i.e. systems constrained to move on a submanifold of R" +1 ) is known as 
d'Alembert's principle of virtual displacements [6, 19]. Essentially, this principle im- 
plies that centripetal forces produce no work along the orbits of the system, a 
property which will be extremely important to us in Section 6. 

Example 5. In the case of the Shahshahani metric, (3.28) becomes <p'{x) = 1/ y/x, 
so we readily obtain the Euclidean coordinates £ a = 2yfx^; this change of vari- 
ables was first considered by Akin [1] and has become known in the literature 
as Akin's transformation [27]. Importantly, the image of X° in this embedding is 
just an open orthant of an n-dimensional sphere of radius 2 (simply note that 
£jg = 4xp, so E/3 = 4), so the Shahshahani metric on the simplex is in all ways 
equivalent to the standard round metric on the sphere. Accordingly, the inertial 
replicator dynamics (I-RD) may be expressed in Euclidean coordinates as: 

L = \z* (»« - 1 E/3 Z} u p) - U« K - nic (e-rd) 

where K = j E/3 represents the system's kinetic energy. 

Example 6. In the case of the log-barrier metric, we will have <p'(x) = 1 fx, so the 
metric's Euclidean coordinates will be £ a = (p(x a ) = logx a . Under this transfor- 
mation, X° will be mapped isometrically to the hypersurface S = {£ 6 R" +1 : 
l,oi < and Yjp e ^ = 1} which is now a closed hypersurface of ]R n+1 that is un- 
bounded in all directions - in stark contrast to the Shahshahani case above. In 
these transformed variables, the log-barrier dynamics (I-LD) then become: 

& = ^ (u« - r~ 2 ^ e 2 ^up) - r~ 2 e^ ^ fr§ - tj^, (E-LD) 

where r 2 = EpX 2 ] = E/s ^ ■ 

We thus deduce an important qualitative difference between the dynamics 
(I-RD) and (I-LD) which is not easy to see when looking at the dynamics on 
the simplex: the inertial replicator dynamics (I-RD) correspond to ordinary New- 
tonian motion on a bounded portion of a sphere, whereas the inertial log-barrier 
dynamics (I-LD) correspond to motion constrained on a closed and unbounded 
hypersurface of ]R' ,+1 . As we shall see in Section 6, this difference will be the 
deciding factor in whether solutions of (ID) exist for all time or not. 

4. Long-term optimization properties 

As a motivating precursor to the game-theoretic analysis of Section 5, we will 
examine here the convergence properties of the inertial dynamics (ID) in opti- 
mization problems defined over polytopes. In this context, the problem's objec- 
tive function will correspond to the potential function of the inertial dynamics 
(ID), and the central question that we seek to answer is whether the local mini- 
mizers of the potential are attracting or not. 

To that end, recall first that the problem of minimizing a function Vq : P — >■ 
R over a polytope P with n + 1 vertices may be pulled back to the unit n- 
dimensional simplex X = A(n + 1) C ]R" +1 simply by considering a linear map 
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which takes the vertices of P to those of X in a bijective fashion. As a result, we 
will focus here on the (smooth) optimization problem: 

minimize V(x) 

subject to xp > 0, ]T)g =0 xp = 1, ^ 

where V : X — >■ IR is a C°°-smooth objective function. 7,8 

Physical intuition and the principle of energy minimization suggest that (ID) 
will indeed be attracted to local minimizers of V; in what follows, we will show 
that this is indeed the case when the system starts with low energy near a proper 
local minimizer of V (i.e. a critical point of V with HesSg(V) y on T^X). 
However, given that we are interested in the long-term convergence properties of 
(ID), our analysis will rest on the well-posedness property: 

The solution orbits x(t) of the dynamics exist for all time. (WP) 

We will address the issue of whether this property holds or not in Section 6; 
for the moment however, it will be much more convenient to retain (WP) as a 
working hypothesis and to condition our presentation on it. With this in mind, 
our first result is the following: 

Proposition 4.1. Let x(t) be a solution trajectory of (ID) which is defined for all t > 0. 
If rj > 0, then lim t ^oo x(t) = 0. 

Before proving Proposition 4.1, it will helpful to establish the following inter- 
mediate result which is of independent interest: 

Lemma 4.2. Let x(t) be a solution trajectory of (ID) which is defined for all t > 0. Then, 
the rate of change of the system's kinetic energy is bounded from above for all t > 0. 

Proof. By differentiating K with respect to time, we readily obtain: 

K = X7 X K = \Vx{x,x) = (V x x,x) = (— grad V — nx,x) 

dV 

where A = sup $f and B = inf{0"(x) : x G (0, 1)}. With A finite and B > (on 

account of the fact that V £ C°° (X) and the Legendre properties of 9 respectively), 
the maximum value of the above expression will be (n + 1)A 2 / (4nB), so K is 
bounded from above. □ 

Proof of Proposition 4.1. Let E(f) = 2||x(£)|| 2 + V(x(t)) be the system's energy at 
time t. Then, by Proposition 3.2, we will have £ = — )/||x|| 2 = —2rjK < 0, which 
shows that E(t) will decrease to some value E* £ IR. This also shows that K is 



■(dV\x) -n\\x\\ 2 = "E/3^ i /3-'7E^^ 



n 

In what follows, it will be convenient to assume that V is actually the restriction of a smooth 
function defined on an open neighborhood of X in R" +1 . To do away with this technicality one simply 
needs to view X as a manifold with boundary and represent V in local coordinates, but this would 
needlessly complicate the issue. We are also assuming C°° smoothness to remain in the category of 
smooth manifolds; our results continue to hold in the C 2 category. 

Q 

Of course, by pulling back Vq to X in this way, the dimensionality of the problem might explode. 
To counter this issue, a better approach would be to derive the inertial dynamics (ID) directly on Pq, 
but since these calculations would take us too far afield, we will postpone them for the future. 
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integrable with f™K(s) ds = ^ (E(0) — E*) < oo. This is a strong indication that 

linif-^oo K(t) = 0, but since there exist positive integrable functions which do not 
converge to as t — >■ oo, our assertion does not yet follow. 

Assume thus that limsup^^ K(t) = 3e > 0. In that case, there exists by 
continuity an increasing sequence of times t n — » oo such that K(t n ) > 2e for all 
n. Accordingly, let s n = sup{f : t < t n and K(t) < e}: since K is integrable and 
non-negative, we will have s n — > oo (because liminf K(t) = 0), so, by descending 
to a subsequence of t n if necessary, we may assume without loss of generality 
that s„_|_x > t n for all n. Hence, if we let = [s n , t n ], we will have: 

K{B)ds>J^J K>eJ^ =1 \J n \, (4.2) 

J In 

which shows that the Lebesgue measure \J n \ of /„ vanishes as t — » oo. Conse- 
quently, by the mean value theorem, it follows that there exists G (s n , t n ) such 
that 

t( K(tn)-K(sn) e 

and since \J n \ — >■ 0, we conclude that limsupK(f) = oo. However, this contradicts 
the boundedness of K(t) established by Lemma 4.2, so K(t) — > 0, and hence 
x(t) — >■ as claimed. □ 

Remark 4.1. The proof technique above may easily be adapted to the Euclidean 
case and thus leads to an alternative proof of the velocity integrability and con- 
vergence part of Theorem 2.1 in [2]; furthermore, even if we add a Hessian-driven 
damping term in (ID) as in [3], the estimate (4.1) remains essentially unchanged 
and our approach may also be used to prove the corresponding claim of Theorem 
2.1 of [3] as well. 

Proposition 4.1 shows that the solution orbits of (ID) slow down, but this does 
not mean that they converge; for instance, they could approach a non-singleton 
invariant set of (ID) with vanishing velocity. That said, if an orbit spends an 
arbitrarily long amount of time in the vicinity of some point q E X, then q must 
be a critical point of V restricted to the subface spanned by supp(^): 

Proposition 4.3. Let x(t) be a solution orbit of (ID) that is defined for all t >0. Assume 
further that for every 5 > and for every T > 0, there exists an interval } of length at 
least T such that max a {\x a (t) — q a \} < 5 for all t G /. Then: 



dV 

dx a 



dV 

dXa 



for all ot,p£ supp(<7). (4.4) 

<? 

For the proof of this proposition, we will need the following preparatory 
lemma: 

Lemma 4.4. Let £: \a,b\ — >■ ]R be a smooth curve in R such that 

g + V£< -m, (4.5) 
for some n > 0, m > and for all t G [a,b\. Then, for all t G [a,b], we will have: 

t]- 1 (Z^ + mrj- 1 ) (l-e-yV-"^ - mn~ x {t-a) if n > 0, 

i(a)(t-a)-\m{t-af ifn=0. 

(4.6) 



?(*)<?(«) + 
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Proof. The case rj — is trivial to dispatch simply by integrating (4.5) twice. On 
the other hand, for rj > 0, if we multiply both sides of (4.5) with exp(rjt) and 
integrate, then we obtain 

£(f) < £(a)e-^- fl ) - mr/- 1 (l - e^"")) , (4.7) 

and our assertion follows by integrating a second time. □ 

Proof of Proposition 4.3. Set up = — J^, [5 =0,...,n, and let a be such that u a (q) < 

Up(q) for all f> £ supp(^); furthermore, assume that u a {q) ^ u 7 (q) for some 7 £ 
supp(^). We will then have w a (^) -i: / j esuppW (0;;(^)/^ / (^))M / j(^) < -m 1 < 
for some m' > 0, and hence, by continuity, we may find some m > such that 

e'^x)- l/1 (u a (x) - £ p (©;;(x)/^(x)) M/S (x)) < m < (4.8) 

for all x £ U<s = {x : max^ — q^\ < 5} and for all sufficiently small S > 
(simply recall that lim XH>0 + 6"(x) = +00 and that 6"(q) > 0). 

That being so, fix 5 > as above, and let M > be such that x a — q a < —5 
whenever the Euclidean coordinates £<* = (p{x a ) of (EC) satisfy £ a < — M. Choose 
also some sufficiently large T > 0; then, by assumption, there exists an interval 
/ = [a, b] with length b — a > T and such that x(t) £ Us for all f e J. Since 
limf_ >O o^a(0 = by Proposition 4.1, we may also assume that the interval / = 
[a, b] is such that £ a (a) is itself sufficiently small (simply note that if x a is bounded 
away from 0, £ a = (p'(x K )x x cannot become arbitrarily large). 

In this manner, the Euclidean presentation (E-ID) of the dynamics readily 
yields 

L<-m + \-^jfYLp &ffl/ty) 2 % - rice <-m- r,tu for all t £ /, (4.9) 

where the second inequality follows from the regularity assumption 8"'(x) < 
0. However, with T large enough and £, a {a) small enough, Lemma 4.4 shows 
that we will have £a(f) < — M for large enough t G J, and hence x{t) 7^ Us, a 
contradiction. We thus conclude that u a (q) = u 7 (q) for all a, 7 £ supp(^), as 
claimed. □ 

Proposition 4.3 shows that if x(t) converges to some q £ X, then q must be a 
restricted critical point of V in the sense of (4.4). More generally, the following 
lemma establishes that any a>limit of (ID) has this property: 

Lemma 4.5. Let x* be an cv-limit of (ID), and let U be a neighborhood of x* in X. Then, 
for every T > 0, there exists an interval } of length at least T such that x(t) £ Ufor all 
tej. ' 

Proof. Fix a neighborhood U of x* in X, and let Us = {x : max^ \xp — x%\ < S} be 

a (5-neighborhood of x* in the L°° norm of with Us H X C U. By assumption, 
there exists an increasing sequence of times t„ — » 00 such that x(£ n ) — > x* , so we 
can take x(t n ) £ LJ^ /2 for all n. Moreover, let t' n — inf{f : t > t„ and x(t) £ Us} 
be the first exit time of x(t) from U$ after t n , and assume ad absurdum that 
t'n — tn < M for some M > and for all n. Then, by descending to a subsequence 
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of t n if necessary, we will have \x K (t' n ) — x a (t n )\ > 8/2 for some a and for all n. 
Hence, by the mean value theorem, we obtain some r n G (t n ,t' n ) such that 

I . / \ I l^«(^n) ^x(tfl) I ^ ^ r 11 //rin\ 

• r ' > — — : for all n, (4-10) 



t' n -t n 2M 

implying in particular that Km sup | x K (t) \ > 5/(2M) > in contradiction to 
Proposition 4.1. We thus conclude that the difference t' n — t n is unbounded, i.e. 
for every 8 > and for every T > 0, there exists an interval / of length at least T 
such that x(t) G Us for all f G /. □ 

Even though the above properties of (ID) are interesting in themselves (e.g. 
compare with the conclusions of Theorem 5.2 for games), for now they will mostly 
serve as stepping stones to the following asymptotic convergence result: 

Theorem 4.6. Assume that the inertial dynamics (ID), rj > 0, are well-posed in the 
sense of (WP), and let q G X be a local minimizer of V with Hess^ V y on TqX. 
Then, if x(t) starts close enough to q and with sufficiently low energy, we will have 
Hrrif-Mx, :x:(f ) = q. 

Proof. Let x* be an o;-limit of x(t). By Lemma 4.5, x(t) will be spending arbitrar- 
ily long time intervals near x* , so Proposition 4.3 shows that x* will satisfy the 
stationarity condition (4.4), viz. 3 a V| x * = dpV\ x * = —u* for all a,jS G supp(x*). 
By the theorem's assumptions, this will allow then us to conclude that q is the 
unique a>limit of x(t), i.e. limf^oo x(t) = q. 

Indeed, if x(t ) starts close enough to q and with sufficiently low energy, Propo- 
sition 3.2 shows that every o;-limit of x(t) must also lie close enough to q (simply 
note that V(x(t)) can never exceed the initial energy £(0) of x{t)); as a result, the 
support of any w-limit of x(t) will contain that of q. However, with Hess^ V y 0, 
V will be strongly convex near q, and since x* itself lies close enough to q, we 
will have 



„ dV_ 
M 5 dx~ R 



(qp-x*p)<V(q)-V(x*)<0, (4.11) 



with equality if and only if x* = q. On the other hand, with supp(^) C supp(x* 
we also get 



bo dx P 



x * /3Gsupp(x») P 



x * pesuppfx*) 

(4.12) 

so x* = q, and our proof is complete. □ 



Remark 4.2. Since the total energy £(f) of the system is decreasing, Theorem 4.6 
implies that whenever x(t) starts close to q with low energy, then x(t) will stay 
close to q and eventually converge to it. This formulation is almost equivalent to 
q being asymptotically stable in (ID), and if q is itself interior, the two statements 
are indeed equivalent. For q G bd(X) however, asymptotic stability is more cum- 
bersome to define because the structure of the phase space of the dynamics (ID) 
changes at every subface of X, so we opted to stay with the simpler formulation 
of Theorem 4.6. 
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Remark 4.3. We should also note here that the positive-definiteness requirement 
(z| Hess^(y)|z) > for every nonzero admissible tangent vector z € T X X can be 
relaxed: all we need is that there be no other point x* near q such that 3«V|;c* = 
dpV\ x * for all a.,f> 6 supp(x*). 

Remark 4.4. Theorem 4.6 is a local convergence result and does not exploit global 
properties of the objective function (such as convexity or geodesic convexity) in 
order to establish global convergence. Even though physical intuition suggests 
that this should be easily possible, the mathematical analysis is quite convoluted 
due to the boundary behavior of the covariant correction term of (ID) (i.e. the 
centripetal force of (E-ID) which constrains the dynamics (E-ID) to S). Since 
the main focus of this paper is the rationality properties of the game-theoretic 
extension of (ID), we will not address this point here and delegate the global 
analysis of (ID) to future work. 

5. Rationality analysis and long-term behavior in games 

In this section, we reconnect with our original objective which was to adapt 
the "heavy ball with friction" dynamics to a game-theoretic setting, and to study 
the rationality properties of the resulting dynamics. 

5.1. Inertial game dynamics. As we mentioned earlier, a foundational interpre- 
tation of the replicator dynamics is that each player looks at the direction of 
individual steepest ascent of his payoff function with respect to the so-called 
Shahshahani metric on the simplex, and then tracks this direction in the hopes of 
maximizing his personal gains [15, 28]. Building on this interpretation, our aim 
in this section will be to derive a class of inertial learning dynamics based on the 
similarly simple premise that the unilateral gradient of each player 's payoff func- 
tion acts as a force on the player's choices and thus determines the acceleration of 
the player's learning trajectory (instead of its velocity). 

Formally, let (S = <S(N,A,u) be a game in normal form and let X = YLk A (At) 
be the game's mixed strategy space. In view of (3.20), if 8 k is a player-specific 
Legendre-type kernel satisfying (L), the unilateral gradient of each player's payoff 
function u k : X — » IR is defined as: 

grad^ u k = (u ka - Yt (®k, h / e kp) u kfi) Eka, (5-1) 

ka 

where, as before, Q' ka is shorthand for 8' k (x ku ), &'^ h denotes the harmonic aggre- 
gate & kh = ( jjf} 1/^) 1 1 an d we have used the fact that 

du k , . , . 

^— = u k {0i;x- k ) = u kct {x). (5.2) 

In other words, the unilateral gradient gradj. u k of u k is simply the gradient of 
u k with respect to the Hessian-Riemannian structure generated by 8 k on X k = 
A(A k ), and evaluated with the strategies of k's opponents kept fixed for the cal- 
culation. 

In this way, the inertial game dynamics that we will focus on the rest of the paper 
will be: 

D 2 x k 



DP 



grad fc u k - t] k x k , (5.3) 
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or, in components: 

= qiT ( u k* ~ Y% (®k,h/ d kp) u kp) 
ka 

- \ ir (C4 - Ej €p*k) - Wu, (IGD) 

where j// c > is a player-specific friction coefficient. Needless to say, in the case 
of the Shahshahani kernel $s( x ) = xlogx and the logarithmic barrier 6i(x) = 
— log x, we obtain respectively the inertial replicator dynamics (I-RD) and the 
inertial log-barrier dynamcis (I-LD), the only difference now being that every 
quantity therein is player-specific. 

Remark 5.1. The dynamics (IGD) are only defined in the interior X° of the game's 
strategy space X, and in view of the blow-up of 6"' near 0, it is not immediately 
apparent how these dynamics may be extended to all of X. When we need such 
an extension, (IGD) will be extended to the (relative) interior of any subface X' 
of X simply by restricting all sums in (IGD) to be taken over the actions a 6 A^, 
k G N, that are supported in X', and by restricting initial velocities x(0) to lie in 
Tw0)X' instead of T X / \X. In this way, every subface X' will be an invariant man- 
ifold of (IGD), a property which in a first order setting essentially characterizes 
the class of imitative dynamics [9] (a terminology which refers to the fact that only 
strategies that are already being employed may be imitated and reproduced). 

5.2. Rationality analysis and long-term behavior. Our aim will now be to ex- 
amine the convergence and stability properties of (IGD) with respect to the Nash 
equilibria of the underlying game. To that end (and to tie our analysis with that 
of Section 4, we first present a preliminary result for potential games, viewed here 
simply as a class of (non-convex) optimization problems defined over products 
of simplices. Without further ado, we have: 

Proposition 5.1. Let = <8(J<f,A, u) be a potential game with potential function V. 
Assume further that the inertial game dynamics (IGD) with rj^ = rj > satisfy the well- 
posedness condition (WP), and let q be an isolated minimizer of V (and, hence, a strict 
Nash equilibrium of <&). If x(t) starts close enough to q and with sufficiently low kinetic 
energy K(0) = \ ||x(0) || 2 , then x(t) will stay close to qfor all t > and will eventually 
converge to q. 

Proof. In the presence of a potential function V as in (2.2), the dynamics (5.3) 
become D 2 x/Dt 2 = — grad V — rjx, x 6 X = Ylk^i-^-k)/ so our claim is proven 
just as in the case of Theorem 4.6: Propositions 4.1 and 4.3 extend effortlessly 
to the case where X is a product of simplices, and even without the positive- 
definiteness condition Hess ( j V y on TqX, it is easy to see that there are no 
other stationary points of (IGD) near an isolated equilibrium of & (cf. Remark 4.3 
following Theorem 4.6). As a result, any trajectory of (IGD) which starts close to 
an isolated equilibrium q of © and always remains in its vicinity will eventually 
converge to q, and since trajectories which start near q and with sufficiently low 
energy have this property, our claim follows. □ 

Remark 5.2. Even though Proposition 5.1 is essentially a reprise of Theorem 4.6, 
it is still important in terms of interpretation because it allows us to view the 
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potential function of a game as the potential of an actual physical system. Specif- 
ically in view of the Euclidean transformation (EC), a potential game can now 
simply be seen as a classical mechanical system which is constrained to move 
on the configuration manifold ^(X) of n*:^^*/ an d which evolves under the 
influence of the potential function V = Vo$ _1 . 

On the other hand, Proposition 5.1 does not say much for general, non-po- 
tential games. In such a context, the most well-known stability and convergence 
result is the folk theorem of evolutionary game theory [15, 27] which states that the 
multi-population replicator equation 

*k a = Xka (U kx ~ «/c) (1-RD) 

has the following properties: 

I. A state is stationary if and only if it is a restricted equilibrium. 
II. If an interior solution orbit converges, its limit is Nash. 

III. If a point is Lyapunov stable, then it is also Nash. 

IV. A point is asymptotically stable if and only if it is a strict equilibrium. 

In the rest of this section, our goal will be to see whether a similar theorem 
may be obtained in the context of the inertial game dynamics (IGD). Of course, 
the second order playing field is fundamentally different from the standard first 
order one, so one cannot expect a verbatim translation of the folk theorem to our 
Newtonian setting: for instance, stationarity and stability cannot be stated simply 
in terms of the players' initial strategy assignments x(0) £ X but one needs to 
also prescribe the strategies' initial growth rate x(0). In spite of these differences, 
we have: 

Theorem 5.2. Let © = <&(N,A, u) be a finite game and let q £ X. Assume further that 
the inertial dynamics (IGD) for © satisfy the well-posedness condition (WP), and let x(t) 
be a solution orbit of (IGD) for rj k > 0. Then: 

I. x(t) = a for all t > if and only if q is a restricted equilibrium of® (i.e. u ka (q) = 

max{u k p(q) : q k p > 0} whenever q ka > 0). 
II. If x{t) is interior and lim^co x(t) = q, then q is a restricted equilibrium of<3. 

III. If every neighborhood U of q in X admits an interior orbit xjj(t) such that xjj(t) £ 
Ufor all t > 0, then q is a restricted equilibrium of©. 

IV. If q is a strict equilibrium of © and x(t) starts close enough to q with sufficiently 
low speed ||x(0) ||, then x{t) remains close to qfor all t > and lim^oo x(t) = q. 

Proof. We begin with stationarity of restricted Nash equilibria. Obviously, given 
how the dynamics (IGD) have been extended to the subfaces of X, it suffices to 
consider interior stationary equilibria. Thus, if q £ X° is Nash, we will have 

"fctO?) = Wfc|5(<7) for a11 a >P e Ac, and hence also u ka (q) = Y^{® k>h f Q kfi )u k fi{q) for 
all a. G A k . Furthermore, with 8"(q) > 0, the velocity-dependent terms of (IGD) 
will also vanish if x kcc (0) — for all a £ A k , so the initial conditions x(0) = q, 
x(0) = 0, imply that x(t) = q for all t > 0. Conversely, if x(t) = q for all time, then 
we will also have x(t) = for all t > 0, and hence u ka (q) = Y^i®'^/ 9 k a)u k p(q) 
for all a £ A k , i.e. q is an equilibrium of ©. 

For Part II of the theorem, note that if an interior trajectory x(t) converges to 
q £ X, then every neighborhood U of q in X admits an interior orbit x\j(t) such 
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that xjj(t) stays in U for all t > 0, so the claim of Part II is subsumed in that 
of Part III. 9 To that end, assume ad absurdum that q has the property described 
above without being a restricted equilibrium, i.e. there exists a G supp^) with 
M A:a( ( ?) < max 0{ M itp(<7)}- As in the proof of Proposition 4.3, 10 let U be a small 
enough neighborhood of q in the L 00 norm of X such that 

eKx)- 1 ' 2 (u kK {x) - (®'l ih {x)/e^{xj) u kp {x)) <m<0 (5.4) 

for all x E U. Then, with x(t) E U for all f > 0, the Euclidean presentation (E-ID) 
of the inertial dynamics (IGD) readily gives 

&* < "™ + \ -7= ®'U%l^%f% - <-m- rjdu for all t > 0, 



kn 

(5.5) 

so, by Lemma 4.4, we obtain f/ Cfl .(f) — > — oo as f — > oo. However, the definition 
(EC) of the Euclidean coordinates ^ implies that x ka (t) — > if £fai(f) — > — oo, 
and since > by assumption, we obtain a contradiction which establishes our 
original claim. 

For the final part of the theorem, assume w.l.o.g. that q = (61,0, ■ • ■ ,£n,o) be a 
strict equilibrium of © (recall also that only vertices of X can be strict equilibria 
of ©). We will show that if x{t) starts at rest (x(0) = 0) and with initial Euclidean 
coordinates ^ ft (0), }i E A* k = A k \ {0}, that are sufficiently close to their lowest 
possible value ^,0 = inf{^>j-(x) : x > 0}, 11 then x(t) — >■ q as f — >■ 00. The 
proof will remain essentially unchanged (albeit more tedious to write down) if 
the (Euclidean) norm of the initial velocity £(0) of the trajectory is bounded by 
some sufficiently small constant 5 > 0, so the theorem will follow by recalling 
that||x(0)|| = |£(0)|. 

Indeed, let U be a neighborhood of q in the L 00 norm of X such that (5.4) 
holds for all x E U and for all ^ E A k = A k \ {0} substituted in place of a. 
Moreover, let U' = <S>(U) be the image of U in the Euclidean coordinates £ = <3?(;e) 
corresponding to 9, and let T(j = inf{f : x(t) E U} = \vA\t : £(f) £ 11'} be 
the first escape time of £(f) = <S>(x(t)) from U'. Assuming T\j < +00 (recall 
that f(f) is assumed to exist for all t > 0), we will have x k ^(x\x) > x^(0) and 
hence ^i^( T u) > f° r some k EJi, }i E A* k ; consequently, there exists some 

t' 6 (0, Ty) such that ^fi( T ') > 0- By the definition of U, we will also have 
*okfi + < ~ m < ror ai l f G (0/ T !i)/ an d hence, with £(0) = 0, the bound 

(4.7) in the proof of Lemma 4.4 readily yields (^(V) < 0, a contradiction. 12 We 
thus conclude that t\j = +00, so we will also have fn, + ^ktku < — m < for all 



Interestingly, this would not be so if we had formulated Part III in terms of Lyapunov stability. 
Note here that Proposition 4.3 does not apply directly because the dynamics (IGD) need not be 
conservative. 

^Recall here that by the definition of the Euclidean coordinates = (f>k( x k^), this condition is 
equivalent to x(t) starting at a small enough neighborhood of q. 

One simply needs to consider the escape time f from a larger neighborhood U of q chosen so 
that if |£fy,(0)l < S for some sufficiently small & > 0, then the bound (4.7) guarantees the existence of 
a non-positive rate of change l,}qi{ r o) f° r some t < f . 
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k G X, ]i £ and for all t > 0. Lemma 4.4 then gives us lim f _>oo = — °°, 

and hence x(f) — > ^. □ 

Theorem 5.2 is our main rationality result for finite games, so some remarks 
are in order: 

Remark 5.3. First off, it will be important to make a comparison of Theorem 5.2 
to the standard first order folk theorem. To that end, we may note the following: 

Part I of Theorem 5.2 is equivalent to the corresponding first order statement 
and basically states that mechanical and restricted equilibria coincide, on the 
other hand, if q is a restricted equilibrium of © and x(0) — q but x(0) 7^ 0, then 
x(t) will clearly not remain at q. 

Part II of the theorem differs from the first order case in that it does not 
disallow convergence to non-Nash stationary profiles. For rj = 0, the reason for 
this behavior is that if a trajectory x(t) starts close to a restricted equilibrium q 
with an initial velocity pointing towards q, then x(t) may escape towards q if there 
is only a vanishingly small force pushing x(t) away from q (compare with what 
happens in the physical world where a projectile may escape Earth's gravitational 
field if given a sufficient boost). That said, we have not been able to find such a 
counterexample for rj > and we conjecture that even a small amount of friction 
prohibits convergence to non-Nash profiles (which, as we shall see in Section 6, 
correspond to points at infinity in the Euclidean dynamics (E-ID) when (IGD) is 
well-posed). 

Part III is at the same time stronger and weaker than its first order analogue. 
On the one hand, the existence of a single interior trajectory that stays close to q 
is a much less stringent requirement than Lyapunov stability; on the other hand, 
and for the same reasons as before, this condition does not suffice to exclude 
non-Nash stationary points of (IGD). 

Part IV is not exactly the same as the corresponding first order statement 
because the dynamics (IGD) are only defined on X°, so we cannot talk about "as- 
ymptotic stability" of a strict equilibrium. Theorem 5.2 shows instead that if x(t) 
starts close to q and with sufficiently low speed ||x(0)|| (or, equivalently kinetic 
energy K(0) = j ||x(0) || 2 ), then x(t) will remain close and eventually converge to 
q. Since this result remains true when restricting (IGD) to any sub face X' of X 
containing q, this can be seen as a form of asymptotic stability of q} 5 

Finally, Theorem 5.2 does not address the converse question of whether (IGD) 
admits asymptotically stable states that are not strict equilibria of 0. This ques- 
tion is intimately tied to the behavior of the corresponding first order dynamics 
Xj. = gradj. Ufc, but since a thorough analysis of these dynamics is beyond the 
scope of this paper, we defer this question to a forthcoming paper [22]. 

Remark 5.4. In the case of the Shahshahani metric and the associated inertial 
replicator dynamics (I-RD), it is also important to compare Theorem 5.2 above to 
Theorem 5.1 of [17], i.e. the folk theorem for the second order replicator dynamics 
(2-RD). In a nutshell, for the dynamics (2-RD), the analysis of [17] shows that: 



This could be formalized by considering the phase space obtained by joining the phase space of 
(IGD) with that of every possible restriction of (IGD) to a subface X' of X, but the gain hardly seems 
worth the effort (see also the relevant remark following Theorem 4.6). 
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I. A state is stationary in (2-RD) if and only if it is a restricted equilibrium of 
0. 

II. If an interior solution orbit converges in (2-RD), then its limit is Nash. 

III. If every neighborhood of a point admits an interior orbit which always stays 
close to the point in question, then this point is Nash. 

IV. If an orbit starts close enough to a strict equilibrium and with sufficiently 
low kinetic energy, then it converges to the strict equilibrium in question; 
conversely, only strict equilibria have this property. 14 ' 15 

A first important difference between the dynamics (I-RD) and (2-RD) is that 
the latter characterize Nash equilibria as o;-limits of interior trajectories, whereas 
the former do not. However, it turns out that there is a much more important 
difference between (I-RD) and (2-RD): Theorem 5.2 is conditional on the well- 
posedness condition (WP), and as we shall see in Section 6, the inertial replicator 
dynamics (I-RD) are not well-posed. 

In our view, this represents an extremely surprising aspect of the dynamics 
(I-RD) which differ from (2-RD) only by a factor 1/2 in the correction term 
x a (x^/x^ — Ejg^g/Xjg). Thanks to Akin's transformation and the presentation 
(E-RD) of (I-RD) in Euclidean coordinates, we are able to provide a geometric 
explanation of this unforeseen behavior in Section 6; that said, this also shows 
that the analysis of higher order dynamics on X can become a subtle affair with 
the end result depending heavily on the geometric considerations at hand. 16 



6. The well-posedness dichotomy 

A key element in the asymptotic analysis of the previous sections is the well- 
posedness property (WP): if the dynamics (ID)/(IGD)are well-posed, then they 
exhibit several compelling long-term properties; otherwise, the very notion of a- 
symptotic analysis breaks down. In this section, we will investigate conditions 
under which the well-posedness condition (WP) holds and we will present two 
important examples that showcase this dichotomy. On the one hand, the inertial 
log-barrier dynamics (I-LD) are well-posed, so the analysis of the previous sec- 
tions applies unchanged; on the other hand (and somewhat surprisingly), the in- 
ertial replicator dynamics (I-RD) are not well-posed - despite the fact that the only 
difference between (I-RD) and the well-behaved second order replicator equation 
(2-RD) is a factor of 1/2 in their centripetal force terms. 

6.1. Inexistence of global solutions in the inertial replicator dynamics. We be- 
gin by showing that the geometric barrier imposed by the blow-up of the Shahsha- 
hani metric does not suffice to keep the solutions of (I-RD) in X for all time, even 



In fact, geometrical considerations are completely absent from [17], but it is easy to see that the 
convergence conditions of [17] essentially boil down to the low energy condition above. 

^Importantly, note that in the second order replicator dynamics (2-RD), the only states that attract 
nearby initial conditions with low energy are the game's strict equilibria. 

l^Even though the dynamics (2-RD) are not derived from metric considerations, they can be de- 
rived from a non-Euclidean connection V on R" +1 with Christoffel symbols f ^ = —$ x fr/x x on 
i.e. twice the Christoffel symbols of the Shahshahani connection. Even though this connection is not 
compatible with the Shahshahani metric, one may still interpret (2-RD) as a covariant system, but 
with the caveat that energy is no longer conserved. 
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in the simplest case of a single agent with 2 actions. Indeed, consider the single- 
player zero game with action set A = {0,1} and payoffs Uq = U\ = 0. Then, 
letting x = Xj = 1 — Xq, the dynamics (I-RD) become 

x = [xjx 2 — x 2 /x — x 1 1 (1 — x)^ = \~Jy~^ 1 ' (^.1) 

so, after the Euclidean change of variables £ = 1\fx, we readily obtain the dyadic 
version of (E-RD) for the zero game: 

I 4 ( 6 - 2 ) 

To solve this equation, let v = £ so that £ = ^ = f|^f = l? f|- By substituting 
in (6.2) and separating variables, we thenget 

dv £ 



v 4- £2 

Letting £o = £(0) and vq = u(0), a further integration yields 



d£. (6.3) 



v 1 4 - f 2 



and solving back for £ = v, we get 



£ = "=IT72\/4-? 2 . (6-5) 



Thus, by separating variables and integrating one last time, we obtain 

arcsin(£/2) = arcsin(fo/2) H ^ , (6.6) 

and a little algebra finally yields the harmonic oscillation: 



I = £ cos +JA- (grin -^L= . (6.7) 




We thus see that for any interior initial position £o £ (0/2) and for any initial 
growth rate Vq =^ 0, f(f) will become negative in finite time, so even in the case 
of the zero game, the solution orbits of (I-RD) may fail to exist for all time. 

In retrospect, this behavior is hardly surprising: after all, the harmonic oscilla- 
tion (6.7) is simply the geodesic equation on the circle S 1 = { (£o/ fl) : £o + £f = 4} 
whose positive quadrant S = {(£o/£l) £ S 1 : £o > 0, £i > 0} is the image of X° 
under the change of variables x i— >■ £ = 2-\/x. More generally, recalling that (E-RD) 
with no forcing (u = 0) is just the geodesic equation on the sphere, it follows that 
its solutions will be represented by great circles, and such trajectories may not be 
contained in any orthant of the sphere for all time. 17 

In view of the above, we see that the failure of well-posedness for the inertial 
replicator dynamics (I-RD) is due to a deep geometric reason: under the Eu- 
clidean change of coordinates (EC), the strategy space of the game is transformed 

i n 

Needless to say, similar conclusions may be drawn in the presence of bounded external forcing 
(i.e. a nonzero game ©) and/ or friction. 
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into a bounded portion of the sphere (endowed with the standard round metric), 
and this bounded space cannot contain geodesies for all time. On the contrary, 
the second order replicator dynamics (2-RD) of [17] do not fall in this geometric 
framework, so, despite their external similarities, it is natural that they behave in 
a very different fashion. 

6.2. Conditions for well-posedness. In view of the above, a fundamental ques- 
tion that emerges is whether any Hessian-Riemannnian metric can give rise to 
a well-posed Newtonian evolution system for games and optimization. Since all 
our analysis in Sections 4 and 5 is conditional on this crucial requirement, we will 
devote the rest of this section to answering this question. 

Reviewing the failure of well-posedness in the case of (I-RD), one expects that if 
the Euclidean embedding (EC) of X° produces a bounded hypersurface of M" +1 , 
then the inertial dynamics (ID) will not be well-posed. Conversely, if the con- 
figuration manifold S = <$>(X°) of the dynamics (E-ID) is a closed (and hence 
unbounded in all directions) hypersurface of ]R" +1 , then a particle moving on S 
under a bounded force will not be able to escape to infinity in finite time. Indeed, 
we have: 

Theorem 6.1. Let g be a Hessian-Riemannnian metric on RTt 1 and let S = <J>(X°) be 
the image ofX° under the Euclidean transformation (EC). If S is a closed hypersurface of 
IR" +1 , then the inertial dynamics (ID) are well-posed on X°: for any xq G X° and for any 
vq e R n+1 , there exists a unique global solution x(t) of (ID) such that x(0) = xq and 
x(0) = vq; conversely, ifS is not closed, then the dynamics (ID) may fail to be well-posed, 
even for u = 0. Finally, subject to obvious changes in notation, the same conclusions also 
hold for the inertial game dynamics (IGD). 



Proof. As indicated by our initial discussion, we will prove this theorem for the 
equivalent Euclidean dynamics (E-ID); we will also only tackle the frictionless 
case rj = 0, the case rj > being entirely similar. Finally, for notational conve- 
nience, the Euclidean inner product will be denoted in what follows by X ■ Y and 
the corresponding norm by | • | . 

With this in mind, let £(£) be a local solution orbit of (E-ID) with initial con- 
ditions £(0) = £o 6 S and £(0) = £o 6 T^ g S; existence and uniqueness of £(f) 
follow from the classical Picard-Lindelof theorem, so assume ad absurdum that 
f (t) only exists up to some maximal time T > 0. Following the physical intuition 
outlined above, let 

F« = ^(" a -^( W')^)' (6 - 8a) 

V ot 

and 

^ = ^L^'/(e^)% (6.8b) 

denote the tangential and centripetal forces of (E-ID) respectively. Since F is a 
weighted difference of bounded quantities, we will have |F(£(f))| < F max for 
some f max > 0; furthermore, as we noted in the remarks following the derivation 
of (E-ID), the centripetal force N will be normal to S. As a result, for all t < T, 
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(a) The inertial replicator dynamics (I-RD). (b) Euclidean presentation (E-RD) of 

(I-RD). 




(c) The inertial log-barrier dynamics (d) Euclidean presentation (E-LD) of 
(I-LD). (I-LD). 

FIGURE 1 . Long-term behavior of the inertial dynamics (ID) and their Euclidean 
presentation (E-ID) for the Shahshahani metric g s (top) and the log-barrier metric 
g L (bottom) - cf. (3.4) and (3.7) respectively. The surfaces to the right depict the 
Nash-Kuiper embedding (EC) of the simplex in R" +1 , and the contours represent 
the level sets of the objective function V: R 3 -» R with V(x,y,z) = (x - 2/3) 2 + 
(y — 1/3) 2 + z 2 . As can be seen in Figs, (a) and (b), the solutions of the inertial 
replicator dynamics (I-RD) end up hitting the boundary of X in finite time, and 
thus fail to optimize V; on the other hand, the solution orbits of (TLD) converge 
globally to the minimum point of V. 

the work of the resultant force F + N along £(£) will be: 

W{t) = ^(F + N) = ^F(Z{s))-Z(s)ds < F max ^ \£(s)\ds < F max £(t), (6.9) 

where £(t) = |f (s)| ds is the (Euclidean) length of the orbit £(s) up to time t. 
On the other hand, with F + N = £, we will also have 

W(t) = f o £(s) ■ £(s) ds = \v 2 {t) - \v\, (6.10) 
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where v(t) = |£(f)| = i(t) is the speed of the trajectory at time t and v$ = |£q|- 
Combining with (6.9), we thus get the differential inequality 

v(t) = l(t) <^v 2 + 2F max £(t), (6.11) 
which, after separating variables and integrating, gives: 

Jv 2 +2F max e(t)-v < (6.12) 

It thus follows that the speed v(t) of the trajectory will be bounded by |£(f)| = 
v (t) < VQt + F max f; similarly, for the total distance travelled by £(£), we will have 
l(t) < v t + \F max t 2 , so |£| and |£| will both be bounded by some ^max and V max 
respectively for all t < T. 

As a result, for any s, f G [0, T) with s < f, we will also have 

\m - £001 < [ \t{r)\dx < v max (t - s), (6.13) 

so if t n — > T is Cauchy, the same will hold for £(f«) as well; hence, with S closed, 
we will have lim^j f (t) = £r 6 S. With £ bounded, we will then also have 

\m - £(s)| < J* \Z(r)\dT < F max (t - s) +Ep/ H(Z(t),£(t))\ dr, (6.14) 

and with sup |£|,sup |^| < oo, it follows that the components of the cen- 
tripetal force will also be bounded: x(t) = <I> -1 (£(t)) remains a positive distance 
away from bd(X) for all t < T, so the weig ht coefficients 6'£/ (6'p 2 of the cen- 
tripetal force N in (6.8b) are bounded, and the same holds for the velocity com- 
ponents £». We will thus have |£(f) — £(s) | < a(t — s) for some a > 0, so the limit 

limf_j.j ^(f) exists and is finite. In this way, if we take (E-ID) with initial conditions 
£(T) = £7 and £(T) = lim f ^x^(f), the Picard-Lindelof theorem shows that the 
original maximal solution £(f) may be extended beyond the maximal integration 
time T, a contradiction. 

For the converse implication, assume that S is not closed in R" +1 , let S denote 
its closure, and let q G S\{S}. Clearly, S will be a closed submanifold-with- 
boundary of R" +1 and the metric induced by the inclusion S ^ IR" +1 on S will 
agree with the one induced by the inclusion S «— > R" +1 on S. Thus, let j(t) be a 
geodesic of S which starts at q with initial velocity vq pointing towards the interior 
of S, and let T > be sufficiently small so that y(T) — p 6 S°. Furthermore, let 
V T = t(F) be the velocity with which 7(f) reaches p; by the invariance of the 
geodesic equation with respect to time reflections, this means that the geodesic 
which starts at p with velocity — vj will reach q at finite time T > with outward- 
pointing velocity — vq. Noting that geodesies on S are simply solutions of (E-ID) 
for u = and t] = 0, and carrying (E-ID) back to X° via the isometry (EC), we 
have shown that (ID) admits a solution which escapes from X° in finite time, i.e. 
(ID) fails to be well-posed if S is not closed. 18 □ 



For general u and )/ > 0, simply let 7(f) be a solution of the dynamics £ = F + N + i.e. (E-ID) 
with —ijg replaced by the momentum-gathering term The time-reflected variant of this equation 
is simply (E-ID), so the rest of the argument follows in the same way. 
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Remark 6.1. It is interesting to note that Theorem 6.1 relates an analytic prop- 
erty of the inertial dynamics (ID) on X° (their well-posedness) to a topological 
property of the geometry of X° (the closedness of the Nash-Kuiper embedding 
(EC) of X°). The asymptotic behavior of the inertial dynamics (ID) is thus tied 
to the underlying geometry of X°, an interdependence which does not exist in 
first order (see also Fig. 1): by contrast, Hessian-Riemannian gradient flows were 
shown in [4] to be well-posed independently of the underlying geometry. 

From a more practical point of view, Theorem 6.1 allows us to verify the well- 
posedness condition (WP) simply by checking that the Euclidean image S of X° 
is closed. Indeed: 

Corollary 6.2. The inertial dynamics (ID)/(IGD) are well-posed if and only if the kernel 
8 of the Hessian-Riemannian structure ofX satisfies f Q yjd"{x) dx = +oo. 

Proof. Simply note that the image S = 5>(X°) of X° under (EC) is bounded (and 
hence, not closed) if and only if Jq <p'{x) dx = Q ^J6"(x) dx < +oo. □ 

In light of the above, we finally see that the analysis of Sections 4 and 5 is not 
vacuous: 

Corollary 6.3. The dynamics (I-LD) and the associated game dynamics (IGD) for the 
log-barrier kernel 6l{x) = — logx are well-posed. 
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